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ABSTRACT 

The National Literacy and the National Numeracy Strategies 
(NLNS) are part of the British government's major education reform 
initiative, an attempt to significantly raise standards of achievement in 
schools. This report is a summary of the first annual report from the Ontario 
Institute for Studies in Education of the University of Toronto (OISE/UT) 
external evaluation of NLNS in England. This evaluation focuses on the 
implementation and the impact of the NLNS, with three key areas': obstacles or 
barriers to implementation; successes and weaknesses; and the emerging 
infrastructure, addressed from the perspective of four levels of the 
education system: national, local education authorities (LEA) , school and 
classroom. This work supplements the evaluation being carried out by the 
Office for Standards in Education (OFSTED) whose focus is the impact of the 
two strategies on the teaching and learning of English and mathematics in the 
primary phase. (PM) 
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Evaluation for Change: 
The OISE/UT Team as 
"Critical Friends" 



t irgc -scale educational reform is 
f difficult to do well. Government 
mandates and promising innovations rarely 
move beyond a few schools or classrooms 
(Elmore. .1996) in a form that improves 
teaching and learning. Fullan (2000) contends 
that the current wave of reform, the most 
ambitious since the 1960s, has arisen, in part, 
because educational change initiatives have a 
poor record of success. Putting ideas into 
practice is clearly a more complex process 
than was expected, especially since the goal is 
not only to establish large-scale reform, but, 
more importantly, to sustain it. 

The National Literacy and the National 
Numeracy Strategies (NLNS) are part of 
the British governments major education 
re fo nil i ni ti a five, an a tte i n p t to si gn i fi can tly 
raise standards of achievement in schools. . 

This report is a summary of the first annual 
report from the Ontario Institute for Studies 
in Education of the University ofToronto 
(OISE/UT) external evaluation team who 
a re sitting in * ‘ fro n t row se ats ” wa to hi n g and 
learning from the implementation of NLNS 
in England. Our role as ‘‘critical friends” is 
to describe NLNS from different perspectives, 
draw connections between the Strategies and 
the international research literature on large- 
scale reform, and identity issues for attention. 
During 1999 the OISE/UT team became 



familiar with the background and rationale 
of die Strategies, made contact with many 
of die key people and agencies, developed 
an understanding of how the Strategies 
are being implemented in England, and 
reviewed pertinent literature. We interviewed 
approximately 200 people, observed over 
20 meetings and conferences, visited 20 
schools and/or LE As, reviewed documents 
and reports, and conducted extensive reviews 
of the international research literature on 
large-scale education reform. These reviews 
(published separately) provided a framework 
for considering NLNS. One of the reviews, 
by Fullan (2000), looked at policy levers at 
the macro level, and examined the history 
of large-scale reform and its return as a major 
force in the past decade. We identifies features 
of the current wave of reforms world wide, 
especially those that appear to be making 
a difference. 

The second review, by Leith wood Jantzi 
and Mascall (2000), focused on the rnicro- 
or school-level conditions that account for 
successful implementation of altered practices 
in schools and classrooms. Taken together, 
these reviews provide a comprehensive 
overview of the state-of-the-art knowledge 
about successful large-scale reform, recognising 
the contributions and roles of both central 
mandates and local action. 
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The Evaluation Framework 



T he OI.SE/UT team developed 

a framework for its work by mapping 
backwards from what NLNS aim to 
accomplish - improvements in pupils* literacy' 
arui numeracy. For such improvements to 
occur, pupils must come into contact with 
° more powerful learning experiences in their 
schools, classrooms and homes. Our 
assumption is that altered practices produce 
these powerful learning experiences and 
thus lead to changes in the literacy 
and numeracy performance of pupils. N LNS 
are policy levers for altered practices to occur. 
All of this activity is occurring within 
England's unique cultural, political , economic 
and educational context. 

Our two conceptual lenses, NLNS as policy 
levers for large-scale reform and NLNS as 
mechanisms to improve teaching and 
learning, allow us to see the Strategies not 
only as a national u scaling up’’ of reform 
initiatives, but also as practices implemented 
at the local level, in classrooms, schools and 
LEAs. Figure 1 is a graphic representation 
Vf the e val u a tion fra me work . 

When we look at NLNS as policy levers for 
large-scale reform, the reference points are 
features of other reform initiatives that are 
underway in many countries and are part 
of the UK effort as well: 



© A vision for the education of pupils; 

© Standards forjudging performance; 

® Curriculum frameworks and other 
instruction a 1 reso u r c es ; 

O Focus on teaching and learning; 

© Coherent and integrated policies; and 

© Accountability and incentives based 
on performance. 

The second lens on NLNS aims to explain 
variation in the success of efforts to improve 
the core technology of schooling - t eaching 
and learning. The literature suggests that these 
variations can be explained, broadly, in terms 
of their influence on: 

© Motivations of educators to improve their 
practices or adopt new ones; 

© Capacities (knowledge and skills) to carry 
out the improvements; and 

© Situations conducive to developing and 
sustaining motivation and capacity for 
productive change. 

There is another important dimension to 
educational change, in addition to the ideas 
included in the framework - timelines. 

How long does it take for sustainable change? 
How much progress can be made in five years 
with the whole nation as the focus of reform? 
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Figure 1: The Framework Guiding the External Evaluation 
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Reflections After Year One (1 999) 



About the Context 

ngland during the 1.990s was ripe 
for educational reform. Although the 
country was engulfed in educational change, 
public confidence in institutions had eroded 
and there was a sense of urgency about the 
need for reform. The Conservative 
government had increased accountability by 
establishing the National Curriculum and the 
National Assessment scheme and the Office 
for Standards in Education (OFSTED). When 
the Labour Party won the election, in 1997 
education, was identified as the number one 
priority, with particular attention to the 
entitlement of those living in disadvantage. 
Prime Minister Blair described the situation 
from the governments perspective: We have 
been good at educating an elite at the top but. . the 
imperative to raise standards for the many in fine 
has been neglected. This focus appears to have 
been widely accepted and endorsed by both 
the public and teachers. In particular, we 
heard many comments about the necessity 
to pay attention to high quality early 
instruction in English and mathematics. 

About NLNS as Policy Levers 
for Large-Scale Reform 

In comparison with other efforts at large-scale 
reform across developed nations, at this time, 
NLNS is among the most comprehensive 
and fully developed. Enhancing literacy and 
numeracy is a clear vision for education in 
England. The National Curriculum and 
NLNS Frameworks give a clear and well- 
developed picture of desired pupil learning, 
with high quality materials and resources to 
support them. There are explicit performance 
standards for pupils and schools, arid 
performance data are used to provide 



assistance and support to schools and 
LEAs that are not meeting the targets. 

The explicit focus on changing teaching 
practices linked to student learning is an 
important feature of NLNS. 1999 saw a 
massive commitment to the infrastructure, 
training and capacity-building to support 
teacher development and strategy 
i mple i ne nta ti on . P rofessi o nally p ro duce d 
packs of materials provide both structure 
and content for training and for subsequent 
implementation. 

Significant new financial resources 
have supported NLNS. In addition, the 
Standards and Effectiveness Unit (SEU) of the 
Department for Education' and Employment 
(DfEE) has worked with other agencies to 
align the Strategies and other policies. The 
revision of the National Curriculum brought 
the English and mathematics requirements 
into line with the Strategies, while the 
program for InitialTeacherTraining is 
being similarly modified. 

NLNS are impressive reform levers that 
have considerable potential to establish an 
infrastructure for lasting change. Both 
support and pressure are embedded in the 
implementation plan with a persistent 
focus on changing practices in schools and 
classrooms. There are, however, some areas 
that deserve attention as the Strategies unfold. 

Particularly in the early part of our inquiry, 
we heard some concerns about lack of 
flexibility in some areas of NLNS: these 
concerns have lessened over time as LEAs and 
schools have deepened their understanding 
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Retlcaiouf After Year One (l 999) 



of the Strategies. Nonetheless, countering 
perceptions of lack of flexibility will remain 
an ongoing challenge. Providing too much 
flexibility always carries the risk, of a return 
to past practices. Achieving and maintaining 
a workable balance requires that teachers 
understand how individual pupil differences 
can be accommodated with in NLNS. 

The apparendy general acceptance of the 
S tra tegi es a c th i s ti me offers a soli d To u n da tion 
for moving to the next stage of implementation 
and for embedding die Strategies into local 
practice in LEAs and schools. While the image 
of schools and LEAs dedicated to literacy and 
numeracy is clear, what is less so is the image 
of them as organisations with the capacities to 
sustain the reform effort. The challenge is to 
embed NLNS in schools and LEAs that have 
developed the ownership, capacity, and other 
characteristics chat allow them to evolve as needs** ~ 
and conditions change. While there is some talk 
about schools as ‘learning organisations” little 
has been done yet to clarity what that means 
and to help schools develop accordingly. 

Clear targets provide teachers, headteachers 
and advisors with reference points for their 
own planning. Ac the same time, the emphasis 
on Key Stage assessments and target setting 
creates a potential risk that scores from a 
single assessment, invested with more 
significance than they may warrant, will 
overshadow the hard thinking that should 
enter into decisions about policy and practice. 
LEAs and schools need to develop “assessment 
literacy” so that they can interpret and use 
performance data wisely. 



About NLNS as Mechanisms 
for Improving Teaching and 
Learning 

NLNS are intended to bring about altered 
practices iri schools and classrooms that will 
result- in improved learning for pupils. To do 
this, the Strategies must influence people’s 
motivation and capacity, as well as the 
situation in which they work. 

NLNS include policies and implementation 
initiatives aimed at fostering motivation (e.g., 
target setting, public statements by politicians, 
and funding for improving literacy and 
numeracy). But motivation is influenced by 
more than external actions. Teachers’ 
motivation is a function of their goals, their 
beliefs about their own capacities, and their 
beliefs about the extent to which they will 
be supported in their schools. In this first year, 
there has been strong support for developing 
the capacity of teachers and consultants to 
teach English, and mathematics differently in. 
primary schools. Teachers and headteachers in 
the schools that we have visited have been 
positive about NLNS.and there seems to be 
high motivation to focus on literacy and 
numeracy. Maintaining motivation through 
the next phase will be particularly important. 
Learning arid using new teaching skills is 
difficult. To stay the course, teachers need to 
believe that making this effort is worthwhile 
and that NLNS will continue to be supported 
as high priorities. 

The resources arid training that are part of 
NLNS are designed to give teachers and others 
the knowledge and skills diey need to provide 
high quality instruction for literacy and 
numeracy to all children, The initial approach 
to developing such capacity relied heavily on 
a “cascade” model, with Regional Directors 
providing training to LEA staff, who then 
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provided training to school leaders, who 
in turn assisted teachers in their schools. But 
N.LNS leaders quickly realised the limitations 
of this approach; a cascade model alone could 
not provide the resources or the in-depth 
learning needed to transform instructional 
capacity. This is especially true with the broad 
scope of NLNS, covering over twenty thousand 
schools across the country. Many teachers in 
these schools need more content knowledge, 
as well as opportunities to acquire those 
classroom practices advocated by the Strategies. 

The evolution of the Strategies has been 
accompanied by considerable expansion in the 
range of ways in which teachers can. increase 
their knowledge and establish new skills. 

Perhaps the most interesting is the development 
(both formal and informal) of networks and 
clusters of teachers working together in local 
regions to discuss and share their perspectives 
and practices. Building the capacity of teachers, 
on a large scale, to provide programmes that result 
in greater competence in literacy and numeracy 
will likely require professional development and 
supervision, for many years to come, as teachers 
continue to re tine the necessary skills and 
understandings. 

Finally, sustainable reform requires supportive 
work in g si tu a ti on s wi th a strong infra stru ct u re , 
for implementation that is flexible enough 
for adjustments to fit with local needs and 
priorities. Many national and LEA initiatives 
have been, designed to enhance the situations, 
contexts, or settings in. which NLNS are to 
be implemented. The national infrastructure 
encourages communication in both directions 
to provide support and to adapt NLNS to 
school and LEA suggestions for modifications 
in. training and implementation. NLNS teams 
are each headed by a National Director, 
supported by a Co-ordinating Director and 
Regional Directors providing training and 



support to the LEAs in the regions. Over 
300 literacy consultants and more dian 320 
numeracy consultants have been appointed in 
LEAs, with half the cost of their salaries borne 
by SEU. For such a large reform initiative, 
NLNS has been remarkably responsive to 
feedback from the field. 

Creating and supporting schools and LEAs 
as workplaces that are congenial to NLNS is 
likely to be a major focus of the next stage 
in implementing the Strategies. Large-scale 
reform initiatives ignore local administrators 
and governors at their peril. LEA 
administrators are very useful partners in 
supporting school-level reform, while their 
indifference can be a major obstacle to 
developing school-level support. In the long 
term, they will have a significant influence on 
institutionalising NLNS. The infrastructure for 
long-term sustainability also needs to include 
continuous conversation with the general 
public and engagement with teacher training 
institutions and higher education. These 
conversations could clarify the nature and 
intent of the Strategies and provide feedback to 
continue the cycle of adjustment and response. 

About Altered Practices 

Across both NLS and NNS our evidence 
suggests that considerable attention is being 
paid to the kinds of classroom practices that 
are consistent with current views of learning 
English and mathematics. Although the '‘best” 
approach to early teaching of English and 
mathematics is still controversial, we conclude 
that NLNS stand as good a chance of 
improving pupils’ literacy and numeracy as 
any other set of policies and practices 
implemented on a large scale. It is too soon 
to tell the extent to which these practices 
are being integrated into the routine work 
of schools and the feasibility of doing so. 
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About Improvement in Pupils' 
Literacy and Numeracy 

The evidence to date seems to indicate 
modest but significant gains being made in 
literacy and numeracy across the country. It 
is early days in the implementation ofNLNS 
anti it would be naive to conclude that the 
instructional and other practices included in 
NLNS were the sole causes of the gains. In 
our view, a large proportion of achievement 
gains to date may well be the result of 
increased motivation, with teachers spending 
more focused time on. literacy and numeracy 
and concentrating their existing capacities oil 
imp roverri e n t i n these areas . Efforts to make 
further gains are likely to bump up against the 
limits of these existing capacities. Continued 
achievement gains will depend on teachers 
learning new skills — a slow and incremental 
process. If this is true, it makes sense to focus 
a very high proportion of future NLNS effort 
on capacity development for teachers and on 
increasing the collective capacity of LEAs 
and schools, so that over the long run they 
will generate their own learning and 
c o n ti n u o us imp rove men t . 



About Timelines 

When the current government took office in 
1997, the targets for literacy and numeracy 
achievement were set for 2002, 5 years later. 
Fullan (2000) suggests that it is possible to 
improve an elementary school in 2 to 3 years, 
a secondary school in 5 to 6 years and a 
school district in 6 to 8 years (depending on 
size). How long it takes to improve schools in 
an entire nation or state is riot known because 
change has never been accomplished on that 
scale before. While there is some evidence 
that the rates of change can be accelerated 
with the intensive use of change knowledge 
(.Earl &' Lee, 1998; Gray, 1999), it remains 
to be seen how much these rates can be 
accelerated NLNS is working toward 
challenging objectives, but with many 
supportive conditions. 
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Conclusions and Challenges 



A s an instance of large-scale reform, 
L.NLNS compares very favourably 
with other such efforts anywhere. But it also 
faces some serious challenges. We offer what 
we believe are . key strengths of the Strategies 
as well as potential challenges as fuel for 
discussion and debate in the next phase of 
planning and implementation. 

Strengths of NLNS 

NLNS is forging new ground in. large-scale 
reform and thus making a significant 
contribution to the international knowledge 
base. In our view, the areas in which NLNS 
design and implementation to date have 
been particularly strong are leadership, policy 
alignment/ coherence, support and pressure, 
communication, resources, and responsiveness 
and adaptability. 

Leadership 

A remarkably strong group of senior leaders 
has come together to develop arid orchestrate 
NLNS. The Head ofSBU and the National 
Directors of the two Strategies bring sets 
of skill and expertise that complement each 
other well as they provide conceptual and 
organisational leadership to the initiative. 
These leaders also have high credibility 
among educators. The Regional Directors 
continue this pattern, advising and supporting 
LEAs and a variety of other educational 
agencies with commitment and skill. 

Policy Alignnient/Coherence 

To an. extent that has rarely been achieved in 
other jurisdictions, other education policies 
have been aligned with NLNS — the National 
Assessments, the National Curriculum, 

OPS TED inspection guidelines, the initial 



teacher-training curriculum and the 
operation of funding formulas. That NLNS 
are a coherent and well-integrated set of 
practices and programmes is a considerable 
accomplishment. The coherence is not yet 
being experienced by all those in LEAs and 
schools, but the central alignment of policies, 
resources and actions is very clear. 

Support and Pressure 

The implementation of NLNS can be seen 
as a tapestry, with support as warp and pressure 
as weft. Sources of support include training, 
resources and consultants, while sources of 
pressure include target setting, funding 
constraints and OFST.ED inspections. 

This complex tapestry is woven together 
in ways that have the potential for '‘deep” 
change to occur. 

Communication 

Communication is a daunting challenge 
in all large and complex reform initiatives, 
particularly one that involves so many diverse 
participants across a range of organisations. 

Iri NLNS, communication has been constant:, 
with senior staff committed to both personal 
and public communication, clarifying 
misunderstandings and trying to 
reinforce consistent messages. Although 
misconirnunications will continue to occur, 
these are usually addressed prompdy. 

Resources 

Unlike many large-scale reform efforts, NLNS 
have been adequately funded, with funds tied 
to specific implementation initiatives. High 
quality curriculum and teaching resources 
have not only made implementation easier, 
they have also helped clarify the nature of the 
practices being advocated by the Strategies. 
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Although the need for financial resources 
and training will continue for some time, 
die nature of such support will shift as 
headteachers and teachers gain confidence 
and competence with new practices. 

Responsiveness and Adaptability 

NLNS have been remarkably adaptable, 
especially given their magnitude. The S.EU 
has paid close attention, to feedback from 
the field and made adjustments based on. the 
response from teachers and headteachers. 

The Regional Directors routinely “test the 
water” in schools and LEAs and incorporate 
changes into their plans. 

Challenges 

Given the magnitude and complexity of 
NLNS, there will always be challenges in 
i m pi e rn en tat i on . In the firs t y ea r o f our 
evaluation, we have identified several areas 
that are likely to require consideration as 
the Strategies unfold. 

Changing Practices is Hard Work - 
Intellectually and Emotionally 

Making the changes required to fully 
implement NLNS has been and will continue 
to be hard work for teachers, heads and LEA 
personnel. They are likely to experience 
frustration and to tire along the way. DfEE has 
paid attention to recognising the efforts and 
successes of teachers, in particular, the national 
teaching awards and the congratulations from 
the government and DfEE on. the Key Stage 
2 results. These efforts will be even more 
important in the next few years as the 
Strategies become more established parts of 
educational practice rather than the exciting 
new initiatives that they are now. 



Motivation is Important, But it is 
Not Enough in the Long Run 

NLNS are complex and robust initiatives with 
the potential to increase levels of literacy and 
numeracy in English schools. 'However, the 
recent improvements in pupil performance 
in Key Stage assessments are likely due to an 
increased sense of urgency and motivation to 
focus on language and mathematics on the 
part of teachers and headteachers. These early 
gains have powerful reinforcement value to 
keep motivation high in the short term. 

As the change process continues, however, 
the demands on schools and teachers will 
evolve considerably. For example, sustaining 
implementation of both NLS and NNS 
will require extensive capacity-building in 
directions not yet anticipated, in addition, 
to a continued commitment on the part 
of headteachers and teachers. Sustaining 
the improvements in pupil learning will 
require continued attention to professional 
development activities, high quality resources 
and opportunities for teachers to practise, in 
order to ensure that the teaching force has the 
knowledge and the skills to make the best use 
of N LNS resources within their unique 
school settings. 

New Teachers are 
a Long Term Investment 

England, like other countries, will, have an. 
influx of new teachers to the profession in 
the next decade. Because these new teachers 
will have the long-term responsibility for 
sustaining the work of NLNS, fostering 
their learning and supporting their confidence 
will pay dividends over time. New teachers 
need to have had a strong foundation in 
the content of language development 
and mathematics learning and in effective 
pedagogy. Although the relationships with 
teacher training institutions were somewhat- 
sparse and somewhat superficial in the early 
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phases of NLNS, the appointment of 
six Regional Directors with specific 
responsibility for initial teacher training is 
a welcome development that should change 
this situation. Links and connections between 
the Strategies and initial teacher training will 
become stronger as these Regional Directors, 
in collaboration with the Teacher Training 
Agency; develop strong relationships and build 
coherence. In addition, NLNS might promote 
and even offer training programmes for newly 
qualified teachers to further increase their 
confidence and competence in teaching 
literacy and numeracy. Eariv-career teachers 
should be at the core of activities designed 
to build a strong culture of optimism and 
continuous improvement. 

Assessment literacy for Wise 
Decisions 

It: is no easy task to find ways to characterise 
or describe the impact of educational reforms. 
Numbers do provide a useful shorthand way 
of describing, communicating and measuring 
what is happening. The challenge is to ensure 
that educators and the public understand 
both the possibilities and limitations of such 
information. Educators and the public need 
to develop “assessment literacy” in order to 
examine student work and performance data, 
make critical sense of it, and to use these 
understandings to alter instruction and other 
aspects of the school to improve student 
1 e a mi n g . Th i s process i s a 1 ready be gi nn i ng 
in. a number of L EAs. Assessment literacy 
not only serves an accountability purpose, 
but also becomes a major professional 
development and school improvement 
strategy. The end result is a sophisticated arid 
powerful contribution to large-scale reform 
goals. As teachers and schools get used to 
working with more data, they become less 
susceptible to naive conclusions based on 



numbers and mote likely to use sound data 
as a basis for improvement planning. 

The Power of Professional Learning 
Communities 

NLNS are unlikely to be sustained in the 
long run unless teachers, headteachers and 
LEA personnel feel a sense of ownership and' 
responsibility for the goals that they represent. 
As the Strategies move beyond the initial 
awareness and implementation challenges, 
it will not be sufficient to have high 
quality training and strong support from 
headteachers. In addition, it will be essential 
to create strong professional learning 
communities at the school level. Teachers’ 
individual knowledge and skills must be put 
to use, not only in individual classrooms, but 
also across the school as a whole. A strong 
professional learning community includes 
clear shared goals for pupil learning, 
collaboration and collective responsibility 
among teachers, reflective professional inquiry 
and opportunities for staff members to 
influence the school’s activities and policies 
(King & Newmann, 2000). As well, sustaining 
NLNS will require that teachers and 
headteachers feel that their programmes 
(and new initiatives they are asked to take 
on) fit together into a coherent whole. 
Perceptions of a stream of unrelated initiati ves 
will not support the kind of sustained, 
collective, problem-solving that is needed 
for continued success. 

DissentingAbices Contribute 
to Clear Thinking 

NLNS are bold and multifaceted initiatives 
that are evolving over time. Throughout the 
early implementation, SEU has been open 
to feedback from the field and .maintained 
a delicate balance between preserving the 
integrity of the Strategies and adapting them 
for local conditions. As people in the LEAs 
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and schools become more familiar and 
comfortable with NLN$, maintaining this 
balance will be more complicated. There will 
be many opinions about the “best way” to 
proceed. S.EU needs to avoid succumbing 
to false certainty and inappropriately rigid 
adherence to the Strategies, while at the same 
time ensuring that the core principles within 
them are not lost. Challenge and debate 
about ideas can help to clarify both intentions 
and approaches during the next few years. 

It would be wise for S.EU to continue to be 
open to this kind of clarification by, as 
Hargreaves and Fullan (1998) suggest, moving 
toward the danger and forging collaborative 



relationships within the educational service 
and the wider community. This means 
establishing new partnerships and reinforcing 
old ones so chat entire systems are the focus 
of change: schools, families, neighbourhoods 
and cities, in concert. In particular, higher 
education can play a powerful role in the 
kind of ongoing analysis and challenge 
that stimulate clear thinking in the effort 
to “get it right.” 

The OISE/UT team is finding this project 
both stimulating and thought-provoking. 

We look forward to embarking on the next 
stage and continuing our association. 
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